Antibiotic Receipt for Pediatric Telemedicine Visits With Primary Care vs Direct-to-Consumer Vendors

Key Points Question How does antibiotic management for pediatric acute respiratory tract infection (ARTI) during telemedicine visits with primary care practitioners (PCPs) compare with visits with commercial direct-to-consumer (DTC) companies? Findings This cross-sectional study of 27 686 children with ARTI visits delivered virtually found that PCP telemedicine visits were less likely than virtual-only DTC telemedicine visits to involve receipt of antibiotics, receipt of a diagnosis that warranted antibiotics, or a follow-up visit or antibiotic fill during the subsequent 2 weeks. The PCP and DTC telemedicine visits had similar rates of antibiotic management that were consistent with the stated diagnosis. Meaning These findings suggest that ARTI antibiotic management via telemedicine differed by context of telemedicine use such that telemedicine policy should account for different models and quality of telemedicine care and support integration of telemedicine within pediatric primary care.


Introduction
Prior to the COVID-19 pandemic, commercial direct-to-consumer (DTC) telemedicine was the form of telemedicine most commonly used for children. 1,2Direct-to-consumer telemedicine is virtual-only care staffed by clinicians not part of the patient's primary care practice, offering care on demand to address common acute concerns, such as acute respiratory tract infections (ARTIs).3][4] Prior research showed substantially higher antibiotic prescribing to children during DTC telemedicine visits compared with in-person visits provided by primary care practitioners (PCPs). 5Those results left unclear whether this difference was associated with modality of care (telemedicine vs in-person) or context of telemedicine care (eg, primary care vs not primary care).
Modality of care (telemedicine vs in-person care) could be associated with these observed differences.Telemedicine limits some examination maneuvers (eg, palpation), requires specific ancillary devices to perform others (eg, tele-otoscopy), and requires additional processes or travel to facilitate diagnostic testing (eg, streptococcal testing). 6Additionally, virtual connections may be dropped or of low quality, potentially reducing the quality of communication. 7Alternatively, context (primary care vs not primary care) could be the more influential factor in prior studies due to differences in clinicians and in continuity.For example, PCPs caring for children are generally pediatricians or family practitioners who have training and experience in in-person pediatric care, while virtual-only practitioners may not have pediatric training or experience caring for children in-person.Additionally, PCPs enter an encounter with prior records, ongoing relationships, and the ability to coordinate in-person follow-up. 8Supporting the idea that these contexts may matter more than modality, 1 prior study 9 found that when the same emergency clinicians practiced in both in-person and telemedicine modalities, their antibiotic prescribing rates were similar in both settings.
Before the COVID-19 pandemic, there was low PCP telemedicine use, making it difficult to directly evaluate whether outcomes differ for telemedicine with PCPs compared with DTC telemedicine.
During the first 18 months of the pandemic, the vast majority of pediatric telemedicine visits were provided by PCPs, as opposed to DTC telemedicine. 3In 1 study 10 during the first year of the COVID-19 pandemic, antibiotic prescribing at PCP telemedicine visits was lower than PCP in-person visits, with high guideline concordance through both modalities.These findings provide support again to the idea that context rather than technology itself may be a key factor in prior findings.
2][13] To better understand antibiotic management in different telemedicine contexts in the current health care landscape, direct comparisons of PCP telemedicine and DTC telemedicine visits are needed.Thus, to fill this knowledge gap, we used 2022 national commercial health plan data to compare matched PCP telemedicine and DTC telemedicine ARTI visits on antibiotic receipt, diagnoses received, guideline-concordant antibiotic management, and follow-up visits.Comparing outcomes for these 2 settings can inform post-public health emergency telemedicine payment and regulatory policy to support telemedicine use that facilitates high-quality care for children.

Study Population
In this cross-sectional analysis, we examined visits of commercially insured children 0 to 17 years of age occurring between January 1, 2022, and December 31, 2022.Data came from the OptumLabs Data Warehouse, which contains deidentified medical and pharmacy claims from a national sample of commercial enrollees.By focusing on 2022, we examined care during a time when PCPs had 2 prior years of telemedicine experience and when in-person visits and ARTI visit rates had rebounded after an early pandemic lull. 4This analysis of deidentified data was determined by the Harvard Medical School Institutional Review Board to be exempt from review and the requirement for informed consent because it was not considered human participants research.This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cross-sectional studies.

Identifying ARTI Index Visits
5][16] We constructed episodes of care that included index ARTI visits (defined as no ARTI encounter in the prior 21 days) and any related follow-up visits (occurring within 14 days of an index visit) to facilitate analysis based on where the first (or index) visit occurred.After constructing episodes, we excluded episodes with less than 1 month of medical and pharmaceutical coverage or lacking demographic data required for matching (4% of episodes) (Figure 1; eMethods in Supplement 1), codiagnoses potentially altering antibiotic prescribing (2% of episodes), 14 multiple visits on the first day (1% of episodes), hospitalization within 1 day (0.2% of episodes), or a COVID-19 diagnosis at the index visit (6% of episodes; due to antibiotic prescribing described elsewhere for these visits). 16

Categorizing ARTI Index Visits
We categorized episodes based on index visit modality (telemedicine vs in-person (eMethods in Supplement 1) and site (eg, PCP, emergency department) using Current Procedural Terminology codes and modifiers, place-of-service codes, provider specialty codes, and revenue codes. 15,16We excluded episodes with index visits that were not via telemedicine and visits to emergency departments, urgent care, or other nonprimary care sites.Each provider identifier for telemedicine visits was categorized based on the volume and proportion of all visits billed under that provider identifier that were conducted via telemedicine (including visits to all ages and for all diagnoses).We recognize that these provider identifier codes could represent a clinic, health system, DTC telemedicine company, or individual clinician.We excluded provider identifiers with fewer than 20 total visits in the claims, due to concerns about the accuracy of telemedicine proportion with few claims.
Of the remaining provider identifiers, we identified those having fewer than 50% of visits conducted via telemedicine.These visits were categorized as PCP telemedicine (eTable 2 in Supplement 1).Among provider identifiers with more than 80% of visits conducted via telemedicine, we identified a distinct cluster of 6 provider identifier codes that had volume of more than 10 000 total visits in 2022, of which greater than 97% were classified as telemedicine.We categorized these provider identifiers as DTC telemedicine.Less than 8% of ARTI telemedicine index visits were billed under a provider identifier code that did not fall into 1 of these 2 classifications; these were excluded from our main analysis (351 provider identifiers, 2531 index ARTI visits).

Primary Outcome
Our primary outcome was whether the index visit resulted in an antibiotic fill.We identified systemic antibiotic fills in pharmacy claims using American Hospital Formulary Service Pharmacologic-Therapeutic Classification System codes. 17If an antibiotic fill occurred on the same day or the ensuing 2 days as an ARTI visit, that antibiotic fill was linked with the visit.If an antibiotic fill could potentially be associated with multiple ARTI visits, we assigned the antibiotic fill to the visit most immediately before the fill.

Secondary Outcomes
In our main analysis, secondary outcomes focused on 3 additional aspects of care: diagnosis received during the index visit, guideline-concordance of antibiotic management during the index visit based on diagnosis received; and follow-up care received after the index visit.Examining the diagnosis received along with the guideline concordance of antibiotic management conditional on diagnosis enabled us to separately examine 2 linked components that may be associated with antibiotic overuse: overdiagnosis and overtreatment.Overdiagnosis of bacterial conditions and unnecessary prescribing for viral illnesses may both contribute to excess antibiotic use. 18Examining follow-up care (including follow-up ARTI visits and antibiotics) enabled us to potentially capture the sequelae of underdiagnosis or misdiagnosis, which could lead to a need for revisits or alternative management.
As already noted, we examined the diagnosis received as an outcome because overdiagnosis of bacterial conditions is 1 mechanism that may contribute to antibiotic overuse.We recognize the possibility of true differences in case mix between sites (ie, symptoms or symptom severity may influence choice of PCP telemedicine vs DTC telemedicine), but we are also cognizant of the subjectivity in diagnosis and the potential for overdiagnoses of bacterial infections as a means of justifying an antibiotic prescription. 19Using ICD-10 codes (eTable 1 in Supplement 1), we categorized each index visit into 1 of 6 primary diagnoses groups.If multiple diagnoses were present, visits were categorized as streptococcal pharyngitis, acute otitis media, or sinusitis if 1 of these "antibiotics may be indicated" diagnoses was present, and as influenza, bronchiolitis or bronchitis, or viral URI only if none of the prior "antibiotics may be indicated" diagnoses were present, consistent with prior studies. 5

JAMA Network Open | Pediatrics
1][22][23] This measure examined antibiotic management for each visit in the context of the stated diagnosis, and so complements the prior measure of diagnosis.We considered antibiotic management to be guideline-concordant if a patient with streptococcal pharyngitis received penicillin or amoxicillin; if a patient with acute otitis media or bacterial sinusitis received amoxicillin or amoxicillin/clavulanic acid or no antibiotic; or if a patient with a viral ARTI diagnosis received no antibiotic (eTable 3 in Supplement 1).Nonconcordance could occur in 3 ways: lack of antibiotics when indicated; use of a nonguideline-concordant antibiotic when a guideline-concordant antibiotic was indicated; and use of an antibiotic when not indicated.
Finally, we examined follow-up care after index ARTI visits as a way to assess for potential misdiagnosis or undertreatment at the index visit.As described in the eMethods in Supplement 1, visits with ARTI diagnoses that occurred within 14 days of an index visit were linked to the index visit as follow-up ARTI visits.Antibiotic fills occurring in the same day or within the subsequent 2 days of these ARTI follow-up visits were identified as follow-up ARTI antibiotic fills.As an alternative more sensitive measure, we also examined any antibiotic fill within 14 days of the index visit but not associated with the index visit itself to capture more broadly any antibiotics prescribed outside of a billed encounter (eg, phone or patient portal communication) or with a new, alternative diagnosis (eMethods in Supplement 1).

Matching
We used exact matching for PCP telemedicine index visits and DTC telemedicine index visits. 24,25sits were matched on the following 6 sociodemographic variables: child sex, age group (0-1, 2-5, 6-11, and 12-17 years), medical complexity status (categorized using the pediatric medical complexity algorithm as no chronic conditions, noncomplex chronic conditions, or complex chronic conditions 26 ), state of residence, and urban-rural status based on the rural-urban commuting area code (metropolitan, micropolitan, small town, and rural). 27Observations were matched many-tomany, pruned, and weighted for balance of covariates.Weights were generated for each matched set (with each set treated as a stratum) to align distributions of matched covariates for DTC telemedicine visits with PCP telemedicine index visits.All analyses of matched visits include application of these weights.

Statistical Analysis
We first compared the frequency distribution of covariates in episodes with PCP telemedicine index visits vs DTC telemedicine index visits before and after matching.Next, we compared our primary outcome (index visit antibiotic receipt) and secondary outcomes (diagnosis received, guidelineconcordant antibiotic management, and ARTI follow-up visits and antibiotics) through separate regressions.To account for clustering of visits within each child, we computed marginal (populationaveraged) outcome proportions using generalized estimating equation log-binomial regression models with telemedicine site as the independent variable.An independent within-child correlation structure was used based on the quasi-likelihood under the independence model criterion.Models used robust standard errors for inference.Predictive marginal proportions and relative risks (RR) comparing PCP telemedicine against DTC telemedicine with 95% CIs were calculated from the models.We determined whether a difference between sites was statistically significant using 95% CI for RR.Statistical significance was defined as a 95% CI excluding 1.We further contextualized results that diagnosis for a given child would not vary with alternative site of presentation, and therefore, we matched visits on diagnosis in addition to sociodemographic characteristics.In our second sensitivity analysis, we allowed for greater subjectivity in diagnosis than the main analysis by including visits that did not have ARTI diagnoses but did have ARTI symptom ICD-10 codes (eg, cough or rhinorrhea) (eTable 1 in Supplement 1) during our construction of episodes of ARTI care, thereby assuming that a child receiving a cough diagnosis at one site could have received a "viral URI" diagnosis at another site.Thus, these 2 sensitivity analyses examined the degree to which our findings varied with more rigid vs relaxed assumptions about the diagnoses that may have been received had the child presented to another site of care.
We also performed 2 additional sensitivity analyses to interrogate the robustness of our results to alternative ways of categorizing the visits occurring with telemedicine practitioners that were excluded in the main analysis (ie, including them with either the DTC or PCP telemedicine categories).
After building these alternative sets of matched index visits and weighting, we repeated the analysis described previously, and present RR obtained through each of these analyses.1).

Main Analysis
The PCP telemedicine index visits were less likely to result in receipt of antibiotics ( providing an antibiotic when not indicated and of not providing an antibiotic when indicated. Nonguideline-concordant antibiotics most commonly used at both sites were azithromycin (46% of nonguideline concordant antibiotics) and cefdinir (34%).For both sites, use of a nonconcordant antibiotic was higher in December 2022 (7.8% and 7.3%) than the remainder of the year, coinciding with an amoxicillin shortage in the US. 28Prior diagnosis codes indicating an antibiotic allergy were found in 7.8% of PCP telemedicine visits and 7.6% of DTC telemedicine visits receiving a nonguideline concordant antibiotic.
The PCP telemedicine index visits had lower rates of follow-up ARTI visits within the ensuing 1 to 2 days (5.0%) compared with DTC telemedicine visits (8.0%; RR, 0.63 [95% CI, 0.56-0.70]).For both sites, when these follow-up visits did occur, they occurred primarily as in-person PCP visits (72% of follow-up visits after DTC and PCP telemedicine).Rates of additional ARTI visits between days 3 and the index visits were also lower for PCP telemedicine index visits (8.2%) compared with DTC telemedicine index visits (9.6%; RR, 0.85 [95% CI, 0.78-0.95]).

Sensitivity Analyses
Sensitivity analyses both matching on diagnosis and incorporating visits with diagnoses limited to ARTI symptoms in episode construction did not significantly change our estimates of the association of PCP vs DTC telemedicine with outcomes (eResults, and eTables 6, 7, and 8 in Supplement 1).
Across our main and sensitivity analyses, we observed no significant difference in nonguideline concordant care by site; the remaining outcomes occurred less frequently at PCP visits than DTC visits in all analyses (Figure 2) (eResults in Supplement 1).Similarly, after categorizing the practitioners who were excluded in the main analysis within either the DTC telemedicine (eTable 9 in Supplement 1) or PCP telemedicine category (eTable 10 in Supplement 1), we observed estimates of the association of PCP vs DTC telemedicine with outcomes similar to our main analysis.

Discussion
This cross-sectional analysis found that compared with pediatric ARTI telemedicine visits delivered via virtual-only DTC telemedicine, PCP telemedicine visits were less likely to involve receipt of antibiotics, receipt of a diagnosis that warranted antibiotics, and a follow-up visit or antibiotic fill during the subsequent 2 weeks.The PCP and DTC telemedicine visits had similar rates of providing antibiotic management consistent with the stated diagnosis.
While prior research has reported increased antibiotic use in pediatric DTC telemedicine visits, it has been unclear whether the higher rate of antibiotic receipt observed with DTC telemedicine visits was associated more with the modality of care (telemedicine vs in-person) or context of telemedicine care (eg, telemedicine used in virtual-only, nonprimary care contexts).Our findings support that the difference is largely associated with the context of telemedicine use (rather than telemedicine itself) given the lower rates of antibiotic receipt for PCP telemedicine visits.We found more specifically that DTC telemedicine visits were more likely to result in a diagnosis for which antibiotics could be indicated but a similar rate of guideline-concordant antibiotic management.It is possible that there was a true difference in the case mix of ARTI conditions that presented to each site (perhaps due to differential parent site selection or practice site triage protocols).Alternatively, prior studies indicate that urgent care clinicians with higher rates of antibiotic prescribing are more likely to provide diagnoses for which antibiotics may be indicated and to prescribe antibiotics, 19,29 highlighting potential subjectivity in diagnoses.

JAMA Network Open | Pediatrics
Antibiotic Prescriptions From Direct-to-Consumer Pediatric Telemedicine Visits Factors that may be associated with increased diagnosis of bacterial conditions and increased antibiotic prescribing may include increased perception of parent expectations of antibiotics, 30 clinician incentives emphasizing throughput or patient satisfaction, 18,31,32 or lack of protocols to transition to in-person care or ensure timely follow-up. 33Each of these factors may be more prominent in DTC telemedicine models in which prior relationships are lacking, the ability to follow-up is limited, and clinician incentives may relate to patient ratings or volume.If more nonpediatricians are caring for children in DTC telemedicine than in PCP telemedicine, that may also be associated with these findings, especially since we found that pediatricians and family physicians had different rates of nonguideline-concordant care even within PCP telemedicine visits.While some of these factors, if found to be contributing, may require solutions specific to telemedicine (eg, developing protocols to smoothly transition to local in-person care), other factors may benefit from outpatient antibiotic stewardship methods.5][36][37][38] Our results also highlight the need to use antibiotic quality metrics that are diagnosis-specific 39,40 (which may mask increased antibiotic use due to overdiagnosis) alongside broader measures 41,42 to monitor and guide improvement in antibiotic management.
Our data suggest that policy in the post-public health emergency era needs to account for the variation within telemedicine models (virtual-only vs integrated) and avoid regulatory and payment strategies that may favor virtual-only DTC telemedicine companies relative to primary care integrated telemedicine care.Our results illustrate that the context of telemedicine care is important for quality; context also influences the feasibility of maintaining telemedicine infrastructure.For example, some decision-makers argue that virtual visits require less overhead costs than in-person Comparison of RRs of select outcomes for primary care practitioner telemedicine visits vs direct-to-consumer telemedicine visits across the main analysis, rigid sensitivity analysis (with visits matched on diagnosis in addition to sociodemographics), and relaxed sensitivity analysis (with visits only matched on sociodemographics and with inclusion of symptomonly visits).NA represents not applicable.
care, which is true for virtual-only vendors but not the case for primary care models offering both in-person and virtual care, which must maintain personnel, space, and triage processes for both types of care. 43Decreased payments for virtual care relative to in-person care may limit the future of integrated telemedicine models by leading primary care practices to avoid investing in telemedicine at all or to feel incentivized to focus on in-person care rather than virtual care when triaging individual visits. 44If primary care practices cannot or choose not to maintain telemedicine infrastructure, families interested in telemedicine turning to virtual-only sites may experience fragmentation and lower quality of care.Policy strategies such as payment parity in fee-for-service models, incentives (financial or otherwise) to primary care practices maintaining telemedicine visits, 45 or value-based payment models 46 may help primary care practices to continue to offer telemedicine integrated within their

Limitations
This study has limitations.As this was an observational analysis of claims data, we lacked clinical information to confirm diagnoses, and therefore performed our primary analysis agnostic to diagnoses and sensitivity analysis specific to diagnoses.We recognize the importance of analyses focused on equity for patients of different racial and ethnic backgrounds; however, patient race, ethnicity, and language information were not available.Future analyses with alternative data should focus on additional patient and practitioner characteristics associated with these outcomes.As this was an observational study, the use of PCP telemedicine and DTC telemedicine was not randomized.
While we sought to account for biases using matching, we cannot account for unmeasured variables.

Conclusions
The results of this cross-sectional study suggest that compared with DTC telemedicine platforms, telemedicine integrated within primary care may perform better in terms of less antibiotic use and reduction of subsequent health care visits.Based on these results, it appears that evolving telemedicine policy may need to account for different models and quality of telemedicine care and should support ongoing integration of telemedicine within pediatric primary care.

Figure 1 .
Figure 1.Flow Diagram of Included Pediatric Acute Respiratory Tract Infection (ARTI) Episodes With Initial Visit Through Either Primary Care Practitioner (PCP) Telemedicine or Direct-to-Consumer (DTC) Telemedicine with additional post hoc analyses (eg, factors associated with increased nonconcordant antibiotic use, sites of follow-up care, rural vs urban residence of child, visits without audio-only Current Procedural Terminology codes, presence of any ICD-10 code indicating antibiotic allergy [Z88, T78] from 2020 through 2022).Analyses were conducted in SAS, version 9.4 (SAS Institute Inc ).In 2 sensitivity analyses, we examined robustness of findings to assumptions about the potential counterfactual diagnosis that patients may have received had they presented elsewhere.In our first sensitivity analysis, we treated diagnosis as more objective than in the main analysis by assuming JAMA Network Open | Pediatrics Antibiotic Prescriptions From Direct-to-Consumer Pediatric Telemedicine Visits JAMA Network Open.2024;7(3):e242359.doi:10.1001/jamanetworkopen.2024.2359(Reprinted) March 14, 2024 5/13 Downloaded from jamanetwork.comby guest on 03/19/2024

Figure 2 .
Figure 2. Comparison of Relative Risks (RR) Obtained Through Main Analysis and Sensitivity Analyses

Table 1 .
Participant Demographics for Unmatched and Matched Index Telemedicine Visits in the Main Analysis Data from 27 686 children (mean [SD] age, 8.9[5.0]years; 13 793 [49.8%] female; and 13 893 [50.2%] male) were included in this study.Prior to matching, the sample included 15 117 ARTI episodes with PCP telemedicine index visits and 15 519 ARTI episodes with DTC telemedicine index visits.In the unmatched sample, the PCP telemedicine index visits contained higher proportions of visits with younger children, with children with chronic conditions, and with children from Northeast and West census regions compared with DTC telemedicine index visits (Table 1).A total of 14 202 JAMA Network Open.2024;7(3):e242359.doi:10.1001/jamanetworkopen.2024.2359(Reprinted) March 14, 2024 6/13 Downloaded from jamanetwork.comby guest on 03/19/2024 PCP telemedicine index visits were matched on child sociodemographic characteristics to 14 627 DTC telemedicine index visits, resulting in balanced sociodemographic covariates in the matched, weighted sample (Table